Testing the cuing hypothesis for the AV speech detection advantage

Seeing the moving face of the talker permits better detection of speech in noise compared to not seeing their face. We report on an experiment that examined the basis of this AV facilitation effect. This work follows up research by [1] and [2] that developed a procedure for demonstrating an AV speech detection effect and [3] that showed that this facilitation occurred regardless of whether participants knew the language of test. In the current experiment we tested to see if AV facilitation occurred because participants were cued to when to pay attention by relatively simply properties of the visual speech of the talker (e.g., when the talker’s mouth opened wide). This cuing idea was tested for two types of auditory and visual information that altered the naturalness of speech but maintained many simply cues. The first alteration was to present the AV stimuli backwards, e.g., (both speech and vision played timereversed). The second used a computer-generated face (Baldi) with synthesised speech. We also tested with a human talker with time-forward presentation. Our findings indicated that AV facilitation only occurred for the time forward human talker presentation; we discuss these results with respect to different types of AudioVisual cuing.